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Outline 



• PA-1 Introduction (short video) 

• PA-1 Roles & Responsibilities & System Providers 

• Gathering inputs from Parent Stakeholders 

• Organizing the project to build the system - (project-centric culture) 

• Project Structure used to cross communicate 

• Defining the system architecture & requirements 

• PA-1 Lifecycle approach 
•Verification approach 

• Conclusions 

NOTE: Lessons learned embedded throughout presentation 
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Presentation Context 



• Slides also intended to serve as a future use reference 

• Slides will tend to have more stand-alone wording 

• Will not delve into specific SE data base tools, Config. Mgmt. processes, etc... 

• PA-l Project did have Config. Mgmt. process, Risk Mgmt. processes, problem reporting process, 
data base tool (for requirements traceability & verification tracking), 

• Focus more on basic approaches & lessons learned rather than specific process & tools 

• Made approach & lessons learned more generalized - apply to most SE challenges 

• Address the human element in implementing a SE approach across a project 
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Orion Launch 
Abort System 



Separation Ring 
. (SepRing)\ 


Launch 

Abort 

System 


Crew 

Module 


0 to 60 mph in 0.28 sec 
0 to 100 mph in 0.42 sec 
500,000 lb thrust 
> 16g for 3 seconds 


A' 



Insert Pad Abort - 1 launch video here!!! 

• From: www.vimeo.com/11631855 




PA-1 Project-Wide Roles & Responsibilities 

(spanned across 4 time zones) 



Lockheed Martin (Prime Contractor) 

• Crew Module Avionics 

• Electrical Ground Support Equip. 

• Mechanical Ground Support Equip. 
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Dryden Flight Research Center 

• Flight Test Article Integration & checkout 

• Systems Engineering Lead * 

• Ground & Flight Operations 

• Developmental Flight Instrumentation 

• Ground Support Equipment 



Orbital (sub to LM) 

• Launch Abort System 


Langley Research Center 

Crew Module Primary Struct. 
Sep. Ring Primary Structure 
Ground Support Equipment 


Johnson Space Center 

Flight Test Office Mgmt. Lead 

Crew Module Parachutes 
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PA-1 Flight Test Article & Providers 




Crew Module 
(CM) Subsystem 

Provider 

Structure 

LaRC (Govt) 

Avionics 

LM (Contractor) 

Instrumentation 

DFRC (Govt) 

Parachutes 

JSC (Govt) 
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Flight Test Office (FTO) Org. Chart for PA-1 

(for reference) 


[• 7777 * 
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Systems Engineering Integration Team (SEIT) Org. Chart for PA- 

(for reference) 
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NASA 

Lockheed Martin 
Joint NASA/LM 


r 


Sys Engineering 


Requirements 

Mgmt 


T&V Mgmt 


Sys Engineer 
WGs 


FT-SEIT 
F TO/PA-1 Lead 
LM Lead 

Deputy for Integration 
PA-1 Lead Eng 
SEIT Support 


T 


Ops Integration 


Gnd Ops Integ 


Flight Ops Integ 


Range Ops Integ 



Operations WGs 


Avionics & S/W 


Avionics Arch. 


Avionics H/W 



Flt/Grnd/Sim 

SW 



T 


Sys Design Grp 



Structures 


Propulsion 


Design 

WGs 


1 


Sys Analysis 


Thermal 




GN&C 



Modeling & Sim 


Analysis 

WGs 
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athering inputs from ALL the Customer Stakeholder 

(What we learned on PA-1) 




Need good representation from your primary customer & system 
stakeholders early in your lifecycle. 

• Besides the primary customer, get inputs from other system stakeholders 

• Anyone than can drive your system requirements 

• i.e. Orion project, Launch site safety, missile treaties, standards, etc... 



“Eureka! The lost valley 
of new customers!" 


Gathering all stakeholders can be more difficult than expected 

• NASA stakeholders commonly spread out across multiple centers, 
agencies & industry partners 

• Cross-talk amongst system stakeholders may be hampered 

• Need 'community organizer' approach to gather stakeholder inputs early | 


If Johnny-Come-Lately's join the system stakeholder forum late: 

• Risk of adding late driving reqts (additional work & schedule delays) 

• Applies to both baselining project reqts & technical review 
entrance / exit criteria. 

• May induce huge delays (& costs) if late inputs result in modifying a major 
contract or redesigning. 
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Finding out what the Customer Needs 

(What we learned on PA-1) 


Initial draft of Mission / Flight Objectives received from 
customer were not mutually understandable. 

• Could have been interpreted differently between the parties 
(project & customer). 


Assumed mutually understandable Mission / Flight Obj. would be 
delivered the first time on a silver platter (not the case) 

• Needed to broker some of their Orion production goals into a flight test realm 

• Solution: We drafted what 'we' thought their needs were 

• Then asked them to tell us where we were wrong. 


Project & customer need to establish a technical rapport 

• Was necessarily tedious & difficult to accomplish 

• Lowered the risk of unknowingly talking past each other 

• Avoided discovering disconnects later in the lifecycle 

• Usually at integration... (too late) 


Commonly understood reference point (Little Joe II) was used 
to directly engage the customer in mutually understandable 
discussions for Mission / Flight Objectives. 



NO. I 
THINK 
YOU'RE 
DISINTER* 
PRETING 
IT. 








V 

Mar w 

[T. 

t "" 
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Organizing the Project to... Build the System 

(What we learned on PA-1) 

Two - Layers to the systems engineering challenge: 

1. Definition, Development, Verification of the system under test 

2. Definition and structure of the support organizations (the people) 

• I was taught... The structure of the project needs to reflect your system 
architecture. 

• Dinesh Verma, Dean School of Systems & Enterprises @ Stevens Inst. Of Tech. 

• Gaps in project structure = gaps in system function & performance. 


Expand on Challenge #2: Coordinate different groups at 

multiple levels across different parts of the country 

• Less of a purely technical effort 

• More of an Engineering / project-based community organizing effort 


Upcoming Slides to address Challenge #2: 

• From: Fragmented Organizational-centric (NASA centers & contractors) cultures 

• To: Single project-centric culture. 

• SE personality type needed to engage communication across project teams 

• Organizational structure reflecting the architecture... for PA-1 project 
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From: Multiple Organizational Cultures 
To: Single Project-centric culture 



What we learned from PA-1.... 

Newly defined project roles & responsibilities, processes 
established across a large (multiple org.) project are not 
instantaneously carried out in a perfect manner. 

It takes some mutual pain (& more time than most like) to 
transition: 

• From: Non-integrated Center & contractor set of cultures, to an... 

• To: Integrated project-centric culture. 

Need influential advocates (community organizers) from each 
org working together. 

• Key agents from each org advocate project-centric culture, approach, 
processes back to their group. 

Need a comprehensive approach / plan to define / develop / 
test system as well as structure project. 

• Each org buys into. 

On PA-1: Became predominantly known as a project-centric 

culture between PDR & CDR 

• Biased opinion of presenter, not scientific assessment 



Non-integrated Center & 
contractor cultures 



Integrated project- 


centric culture 
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From: Multiple Organizational Cultures 
To: Single Project-centric culture 



What we learned from PA-1.... (Cont.) 

Set up communication forums / hubs for technical cross talk 

• Roll call & status from al[ discipline leads 

Need team-wide collaborative web environment 

• One place to find the latest document version & related info. 

• Very helpful with coordinating & tracking verification 

• Sometimes difficult to achieve 

• Organizational web security standards 

• Contractual / proprietary issues among project partners 

Project & Team-wide meeting calendars were essential 

• One reference point for team meetings. 

Team social events away from PowerPoint venues were beneficial ! 



Flight Test Office had direct control over most project teams.... 

* But only had 'influence' over some project teams 

• Could not rely on direct (contractual) authority 

• Rely even more on community organizing skills to engage 
these groups and... the mgmt structure above them. 

• Dedicate person within project to work directly with 
'influence-only' partners. 
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From: Multiple Organizational Cultures 
To: Single Project-centric culture 



What we learned from PA-1.... (Cont.) 


Watch out for the typical engineering drill-down mentality 

• Til focus on my part, you focus on yours..." 

• Most engineers delight in avoiding the human interaction aspect of 
engineering and desire to focus solely on the product itself. 

• Reiterate: Engrs. need to think & talk across org. & system boundaries 


Project communication gaps swarm around Lone Rangers 

• Project Community Organizers need to spot & close these gaps 


Assume cross-functional project communication will fail 
at some point unless: 

• Key disciplines across project are proactively & directly 
engaged regularly... throughout lifecycle 

• "Unless everyone who needs to know does know, ... somebody 
somewhere will foul up" 

• Eberhardt Rechtin, 1997, The Art of System Architecting 
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From: Multiple Organizational Cultures 
To: Single Project-centric culture 



What we learned from PA-1.... (Cont.) 


Some PA-l evidence of a project-centric culture: 

Unsolicited comment from a Lockheed avionics engineer to a 

NASA systems engineer (PA-1 post-flight ' social ' event): 

• "It would be a shame to break up this team... For example, whenever 
I wanted, I could just pick up the phone and talk directly to the (LaRC) 
structures lead to see how possible changes affect us both." 
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Valuable Systems Engineering traits when Organizing a Project 



nhQ * 


Systems Engineering / Community Organizer traits: 

* Don't necessarily have to be overly social 

* However SE'ers need to: 

• Engage a wide variety of personality types across the project 

• Be very approachable 

• Recognize communication gaps, for example: 

• Only hear repeated concerns on only one side of the story / issue. 

• No clear way for groups to engage each other 

• Carry forward concerns / issues over communication barriers 

• Be organized... beyond just yourself 

• Also be an organizer 

• Participate in regular forums that promote cross-talk 

* Value added if above qualities apply to project leads as well. 

• Others on the project can help organize, but.... 

• It's the SE's job to assure the organizational structure supports the architecture 
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Valuable Systems Engineering traits when Organizing a Project 




When project leads are not a fan of NPR 7123.1a 

• Don't confront them as if you're the NPR police... 

• Win them over by asking, "How can we best make ' ' clear to 

others within the project?" 

• This is how they can meet the intent of NPR 7123.1a .... w/o them 
knowing it (sneaky...) 

• In the background you can check off the NPR 7123.1a check-list 


Y 



Some project leads may not fully understand Systems Engineering 

• Help ghost-write their requirements if necessary 

• This was done for 1 module and 1 subsystem on PA-1 
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Project structure used to establish project-centric culture 

(for PA-1) 



Flight Test Engineering 
Review Board (ERB) 

Orion (Parent) Org 

Flight Test Office Org. Technical 

Decisions 

Engineering Review 
Team (ERT) 

f 


Flight Test Office 




LAS Module IPT ■■ GSE IPT 


SEIT 


Avionics Operations System 

Design 

Positions were discipline & deliverable specific, not center specific. 


System 

Analysis 


Systems 

Engineering 


Crew Module/ 
SepRingIPT 


Launch 
Facilities IPT 


Test & Verification 
Control Panel (T&VCP) 

I 

Flight Test Panel 

Programmatic 

Decisions 

(cosVschedule) 


Can't guarantee this is the best way to organize, but: 

• It was clear and understandable to the team... which 
compensates for a lot. 


Parent Org (Orion) Structure: 

• ERB: Technical decisions 
impacting parent org 

• T&V Control Panel: Cost / 
schedule decisions impacting 
parent org. 

FTO Org. Structure: 

• ERT: Tech, decisions w/in FTO 

• Fit. Test Panel: Cost / schedule 
decisions w/in FTO 

• 4 Module level IPT's 

• SEIT (5 branches) 

1. Systems Eng. 

2. Avionics (largest & most 
complex subsystem) 

3. Operations 

4. System Design 

5. System Analysis 
• Met every week 
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Defining the Architecture 



* "If social cooperation is required, the way in which a system is 
implemented and introduced must be an integral part of its 
architecture." 

• Rechtin, E. "Systems Architecting, Creating & Building Complex Systems" 
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Defining the Architecture (Cont.) 



* Before we generated system requirements, we defined the architecture 





a 9 


S3 E 


□ □□ 


CO CL 


Control Room 


System Level R equirements 

Allocation to ModuleVLevel Requirements 


External Interface 
l Requirements J 

Control Room 
Gnd Support Equip. 
Launch Facilities 
Range 


\Sep Ring Module^ VCrewModuley 

— v 


LAS Modu e 


Allocation to Subsystem^^Level Requirements 


CM Subsystems 


O oi 


Structure Power Chutes Avionics 


Therm. 
Cond. Inst. 


Definition of 
architecture helped: 

• Define spec, tree 
hierarchy 

• Define requirement 
allocation categories 

• Define boundaries of 
elements within system 

• Next slide... looked at 
system elements from 
3-views 
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Example of 3-View Architecture Definition for Crew Module 

(This approach was used across the system) 





Functional Vi 


Viev^ 


C = Thermal Cond. 


Sep Ring 


C * Tlwrmal Co«d 
F * FtuMS <RCS) 


S*pR)OQ 


Physical View 


Took global perspective of 
system elements: 

• Functional View 

• Dev. & Op. Phases 

• Functional Modes 

• Sample slides shown 

• Interface View 

• External Interfaces 

• Sample slides shown 

• Physical View 

• High Level Physical 
Attributes 

• More detailed 
attributes (weight, C.G., 
Moments of Inertia, 
OML) in a separate 
Geometry & Mass 
Properties doc. 

• No sample slide 


Page 23 


Actual 'Phase' Chart shown @ PA-1 SRR 

(From Functional View) 




Transit Phase 
Stack 


Move 



Unstack 


Crew Module Phases 


Ground Ops Phase 


Integration/ 

Checkout 


Service 


Launch Pad 
Integration 


Pre-Launch 


Flight Phase 


Launch 


• Defined Transit, Ground Ops & Flight 
phases were used to derive supporting 
requirements for the Crew Module. 


Abort 


Recovery 
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Actual 'Phase' Chart shown @ PA-1 SRR (Cont.) 

(From Functional View) 




Transit Phase 


Stack 


Move 


Swing 


Weigh 


Unstack 


Crew Module Phases 


Ground Ops Phase 


Integration/ 

Checkout 


Service 


Req. # 

Requ 

irement Name 

FTA-S-01 

Ground Handling Factor of Safety 

FTA-S-02 

Shipping Factor of Safety 

FTA-S-05 

Supportable in Vertical Position 

FTA-S-06 

Lifting Point for Vert. & Horiz. Handling 

FTA-S-07 

Crane lift 

FTA-S-08 

Transportability 

CM-S-13 

Vertical Stacking 

1 by Crane 


requirements at their SRR (is it really an SRR then?): 

• Time constraints are understandable, but: 

• Example above is proof it's possible to review 
^^j^cjuh^ment^^^jDamjDhmsecMeve^^RR^^^^ 


Recovery 
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Actual 'Functional Mode' Chart shown @ PA-1 SRR 

(From Functional View) 



FT A Mode Sequence 


Crew Module 
Funct. Modes 




1 

Abort Enable Criteria met 
AND 

User/Auto Command 

User Cmd AND 
Ul not attached 




Abort Criteria met 


AND 

Pad Abort 


User/Auto Command 

Failure 




User 

Cmd 



Altitude/Time 


Criteria Met 


OR User 


Command 


Reqt # 

Requirement 

Steps 

FTA-F-07 

Startup 

1 

FTA-F-24 

Init to III 

1 to 8 

FTA-F-08 

Abort Inhibit 

2 

FTA-F-09 

Init to Abort Inhibit 

1 to 2 

FTA-F-10 

Al to AE 

2 to 3 

FTA-F-1 1 

AE to Al 

3 to 2 

FTA-F-12 

Al to Shutdown 

2 to 6 

FTA-F-1 3 

Failed Launch SD 

2 to 6 


Reqt # 

Requirement 

Steps 

FTA-F-02 

Abort 

3 to 4 

FTA-F-1 4 

Failed Abort SD 

3 to 7 to 6 

4 to 7 to 6 

FTA-F-03 

Recovery 

4 to 5 

FTA-F-1 5 

Shutdown 

5 to 6 

FTA-F-23 

Init to SD 

1 to 6 

FTA-F-25 

Ul to SD 

8 to 6 

FTA-F-28 

Ul to Sim 

8 to 9 

FTA-F-26 

Sim to Ul 

9 to 8 

FTA-F-29 

Ulto Al 

8 to 2 


• Paraphrased versions of 
the requirements were 
used to walk reviewers thru 
the requirements at SRR in 
an expedient manner. 
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Actual 'External Interface' Chart shown @ PA-1 SRR 

(From Interface View) 

Crew Module External Interfaces 



r+ ++< 

ORION 

Kk 


User Interface 


C = Thermal Cond. 


LAS 


M S 


Sep Ring 


M, S, F 

P, c 


Launch Tower* 



M 

P, C, F 

M 


S 

p,c 


GSE** 



Ground 

Handling 

And 

Transportation 

Equipment 



Launch Pad** 


Used to get stakeholder agreement on external interface types 


48 
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Actual 'External Interface' Chart shown @ PA-1 SRR (Cont.) 

(From Interface View) 




Crew Module - Range Interface 




LAS 

M, S, F 
P, C 

Launch Tower* 

User Interface 


S 

M S 

■ f 




• Paraphrased versions of the requirements were used to walk 
reviewers thru the requirements at SRR in an expedient manner. 
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Top Tier of PA-1 Spec Tree 

(For Reference) 




Page 29 



















Defined System & Instrumentation 
Sensors in a parallel manner 



Mission Objectives drove 
the system-wide design 
Mission Objective: ..I 
demonstrate ▼ 
satisfactory perf. & 
operation of the LAS. 


Flight Objectives Drove Master 


Mission & Flight 

1 

nF! 1 Objective: Determine stability char, of LAS+CM 

Objectives 



configuration during a pad abort 


Standard 
allocation to 
lower level 
requirements 


\7 


System 

Requirements 

Document 


Module A 


Module B 


Module C 





Subsystem A 




Subsystem B 


Subsystem C 


V 



• Measure Of Performance (MOP): 

• Evaluate LAV attitude (including flight path 
angle, ijj, 0, cj)) 

• Evaluation Criteria: 

• LAV dynamics compared to 6-DOF 
simulation, adjusting for day-of-flight 
conditions 

Required Parameters: 

• LAV position, velocity, acceleration, attitude, 
angular rates, angle of attack, sideslip, 
estimated thrust from abort motor, day-of- 
flight winds, and atmospheric conditions 
derived from on-board measurements. 


Master 

Measurement 
List (MML) 


• LS041V: Z-axis acceleration .... 

• LSO.... 
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Pad Abort 1 Review Lifecycle 


* "Before proceeding too far, pause & reflect! Cool 
off periodically and seek an independent review" 

• Douglas R. King, 1991 



• "If you think your design is perfect, it's only 
because you haven't shown it to someone else." 
• Harry Hillaker, 1993 
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Pad Abort 1 Review Lifecycle (Cont.) 
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Pad Abort 1 Review Lifecycle (Cont.) 



Technical Review Entrance / Exit criteria tailored from NPR 7123.1a 
Appendix G 

• Approved by customer well before each review 

• Resulted in mutually clear expectations for each review early-on 

Early coordination with customer helped achieved timely buy-off 
of review approach 

• Increased likelihood of reviews meeting customer expectations 

• Without early coordination: Increase risk of surprising customers at the 
review ("... can't proceed to the next phase until.... you do A, B, C, etc...") 


• WARNING: Customer may still change their mind on review criteria 

• But, baseline criteria will help justify impacts 

STTR approach used to approve procurement & basic design of CM 

Primary Structure before PDR (yes, I said PDR). 

• Used only if: 

• Risk of expediting project is lower than the schedule risk of waiting 
for the review 

• Have a well established risk mgmt system to track / update risk 
mitigations (i.e. workable retro-fits for increased loads from 
downstream analysis). 
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Pad Abort 1 Review Lifecycle (Cont.) 

^^^^^JWhatw^eamec^n 



Entrance / Exit criteria used to define presentation template for 

each subsystem at each technical review. 

• Provided consistency for each subsystem presentation 

• Made it easier to define subsystem readiness gaps (issues) & go fwd plans 

• Reduces chance of overlooking something important across system 




Tailoring of entrance / exit criteria was / is key: 

• I was taught... Strictly following a text book approach for systems engineering on a 
project would practically guarantee failure . 

• Dinesh Verma, Dean School of Systems & Enterprises @ Stevens Inst. Of Tech 

• Do NOT deny engineering judgment from past pain 


Go forward plan 



Examples of "tailored" subsystem presentation templates shown on next 2 slides for PDR. 
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Example of Subsystem presentation 
outline / template for PDR (PTR-2) 

• Entrance Criteria - tailored from NPR 7123.1a for your subsystem 

• Schedule - Subset of the master schedule for your particular subsystem / deliverables 

• Document/s Status - Self explanatory 

• Driving Requirements - Shows requirements that are causing your design to be 'what it is/ 

• Safety - Hazards pertaining to your particular subsystem 

• External Interfaces - Summary of interfaces external to your subsystem 

• Design Concept - Block diagrams, Sketches, Drawing trees, Analysis 

• T&V Approach - Basic description of Test approach and how requirements will be verified. 

• Issues & Resolutions - Identify open issues and a plan on how they will be resolved. 

• Go Forward Plan - Path to CDR 

• Exit Criteria - tailored from NPR 7123.1a for your subsystem 

- Resulted in reviewers knowing expected topics for each subsystem. 

- Enabled reviewers to consistently compare subsystem readiness across the system. 

- Made it easier for project to pro-actively define go-forward plans for subsystem 'issues' 
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Example of Subsystem Entrance / Exit 
Criteria template for PDR (PTR-2) 



PTR-2 Subsystem Level Entry Criteria 

Slide 

Preliminary subsystem specs for each H/W & S/W Cl 


Draft Subsystem Interface Requirements Docs 


Draft Interface Control Documents 


Design / Analysis Documentation 


Engineering Drawing Trees 


T&V Planning 



• Consistently showed reviewers 'how' each 
subsystem met its share of the system-wide 
entrance / exit criteria. 

• If template not used... could result in 
inconsistent coverage from subsystem to 
subsystem. 

• Reviewers may conclude project 
coordination is 

• Warning flags 


inconsistent^ 
go up Hp 


PTR-2 Subsystem Exit Criteria 

Evidence 

Slide 

Subsystem requirements defined & trace to parents & 
are allocated to components & external subsystems 

• Driving Requirements show traceability 

• Requirement allocations are in specs 


Subsystem Level designs exist and are consistent 
with their corresponding requirements set 

• Design spec complete with TBD/Rs 

• Design drawings % complete 


Subsystem interfaces identified and are consistent 
with their corresponding subsystem design maturity 

• IRD / ICD’s with TBDs / TBRs 


Project risks identified & mitigation strategies defined 

Project risk #’s in IRMA risk database 


T&V approach is adequate to proceed 

Verification methods identified & test 


S&MA adequately addressed in the preliminary 
design & the preliminary design-based S&MA 
requirements & approach have been approved 

Hazard report #’s & referenced S&MA 
analysis 
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Verification 

(What we learned from PA-1) 



Early-on: 

• Believed defining & implementing workable requirements would be the greater challenge 

• Foregone conclusion that the easier task would be to record the verification of those same 
requirements later in the lifecycle. (WRONG) 


Looking-back: 

* Experience taught us: 

• No tasks can realistically be categorized as significantly easier through lifecycle 

• Complexity of coordinating the human element of requirements verification 
comparable to human element challenge of implementing those same requirements 
earlier in the lifecycle. 

• i.e. Coordinating latest versions of test results & analysis at each associated level 
while briefing burn-down status 

• Next slide touches on contributors to this challenge 
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Verification Planning 

(What we learned from PA-1) 



Requirements & 
D evelopment activities 
System 
Level 

Module 
Level 


Integration & 

Verification Activities 

Verification planning activities 


Verification Planning Activities: 

• Strong correlation within module & 
subsystem verification efforts 

• Gaps in correlating Module & Subsystem 
verifications with System level verif. activities 

• Leads busy implementing requirements & 
design early in lifecycle 

• Less time to tie all levels in system 
verification planning 

• Made for more work later in the lifecycle to 
correlate latest (under the gun). 



Lesson Learned: 

• Where ever possible: Complete system verification planning 
efforts with module & subsystem leads earlier in the lifecycle 
• Set up more direct 'check-list' of tasks to reduce avoidable 
system-wide review & analysis later in the lifecycle 
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Actual PA-1 Subsystem Verification Chart briefed to Mgmt. 


What we learned from PA-1... 



2.000 


1.800 


1.600 

1.100 

1.200 

1.000 


800 


600 


Most subsystem verifications were more straight fwd 
compared to module & system level verifications 
• Module / System: more integrated analysis / test results). 


{+ 11 ; 


Some subsystem & module requirements were held 
in verification "purgatory 7 (incrementally verified 
multiple times as integration scope expanded) 


Subsystem / module reqt verification sent to 
heaven ^ when final integrated 
verification activity was completed. 


(+ii) 




i CLOSED 
i OPEN 



V 

Structures 

Avionics 

Mechanisms 

DFI 

EGSE 

MGSE 

Total 

Software 

m CLOSED 

358 

680 

78 

107 

201 

429 

1 853 

528 

■ OPEN 

0 

2 

0 

6 

0 

0 

8 

0 



99.5% 
Comp I 


etc 


129 subsystem requirements have been closed since January 
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Actual PA-1 Module Level Verification Status Chart briefed to Mgmt. 


What we learned from PA-1... 


500 

450 

400 

350 

300 

250 

200 

150 

100 

50 


Two kinds of module level verifications: 

1. Purely a subsystem child roll-up 

• i.e. environmental requirements 

2. Child roll-up with some form of integrated analysis: 

• More paperwork used here to tie-in all combination 
of integrated analysis and test results. 

• Use community organizing skills to assure module, 
subsystem, analysis & integration leads are talking. 








CLOSED 
l OPENED 


u 

LAS-CM 

Interface 

CM 

CM -SR 
Interface 

NASA GSE 

LM GSE 

MOF 

Total 

■ CLOSED 

54 

157 

10 

47 

151 

75 

494 

■ OPENED 

4 

21 

0 

0 

0 

0 

25 


95% Complete 
(+ 1 %) 
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Actual Module Reqts. Burn-down Chart Briefed to Mgmt. 




Total Module- level Verifications 

522 


550 


500 


450 


400 


350 


What we learned from PA-1... 
Range of 
mgmt 
feedback if 
project is 
ahead or 
behind the 
verification 
burn-down 




Constructive 

feedback... 



U 1 
•$ 

r r r 

“ n \ n% n \ n% ' 

r 

r 

\<b 

V 

r 

v*£ 

V 


r 
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r 

Y*V> 

V 

& 

r 

$ 

i 


& 


1/4 

i/u 

1/18 

1/25 

2/1 

2/8 

2/15 

2/22 

3/1 

3/8 

3/15 

3/22 

3/29 

4/5 

4/12 

4/19 

4/26 

Verifications Plan 

308 

308 

306 

294 

172 

152 

87 

76 

65 

55 

45 

35 

25 

15 

7 

0 

0 

— — Verifications Projected 

308 

308 

306 

294 

172 

111 

97 

74 

73 

71 

66 

31 

31 

25 

7 

6 

0 

Verifications Actual 

308 

308 

306 

294 

172 

111 

97 

74 

73 

71 

66 

31 

31 

25 

7 




7 unverified requirements 

- 6 to close after FTRR (waiting on future activities) 
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Actual System Reqts Verification Burndown briefed to Mgmt. 



Total System-Level Verifications 
94 


550 


* System Level Burn-down was more of a gradual slope 

• Factored in a buffer turn-around time for completion of 
paperwork after successful verification (A, T, I, D) activities. 


If you're behind the burn-down profile... 

* Have a credible story for mgmt on how you'll still meet fit. date 

• Example: "Of the 11 reqts above the burn-down, 10 were 
successfully tested, but are awaiting final approval our project review 
board, which is tomorrow." 


FTA Post- 
Stack 
Functional 



^ ^ # # 


4 ? 


Jr 






\V 


& 


& 


tv\ 


& jr 



1/4 

i/ii 

1/18 

1/25 

2/1 

2/8 

2/15 

2/22 

3/1 

3/8 

3/15 

3/22 

3/29 

4/5 

4/12 

4/19 

4/26 

System Reqt Plan 

88 

88 

88 

88 

84 

82 

79 

75 

70 

64 

56 

46 

35 

24 

12 

0 

0 

System Reqt Actual 

88 

88 

88 

88 

84 

83 

83 

79 

70 

64 

59 

56 

49 

39 

23 
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Conclusions & Perspectives Gained 


• Get engaged early with ALL of your parent stakeholders - Establish technical rapport 

• Importance of looking at organic parts of the project supporting the system. 

• i.e. Project organization, processes, various disciplines, human nature 

• Needs to be worked in parallel with defining the system 

• Reflects the architecture 

• The more clear things can be made within the team, the more achievable a project- 
centric culture will be. 

• Single reference points for (defined preferably in a collaborative web environment): 

• Project & Team meetings (with charters) 

• Technical & Project decision process - For decisions affecting project or technical baselines 

• Schedule 

• Organizational structure & roles / responsibilities 

• Risk Mgmt 

• Configuration Mgmt 

• Problem reporting & resolution 

• Technical Review approach & entrance / exit criteria 

• Key project & engineering documents 

• Verification Planning 

• To get a large group of individuals in different orgs across the country to develop a 

cohesive system... 

• Takes more than a sound SE approach 

• It also requires a human interaction mindset that is not intuitive to most engineers. 
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Conclusions & Perspectives Gained (cont.) 


• Get stakeholder buy-in of architecture definition before deriving system requirements 
• Derive system requirements from architecture definition. 


• Have a template for subsystem presenters at technical reviews tailored from NPR 
7123.1a entrance / exit criteria 




• Verification coordination will sneak up on you if not thoroughly completed early-on 
• Correlate Module & Subsystem verifications with System level verification activities early-on 
• Reduces frantic scrambling around later in the lifecycle 


Side Notes: 

• PA-l project passed 2010 NASA OCE Systems Engineering audit 

• 2011 NASA Systems Engineering (SE) Excellence awarded to the Orion Pad Abort-1 SE Team 
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Conclusions & Perspectives Gained (Cont.) 

Systems Engineer Triangle 
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Community Organizer Mindset 
(Assure org reflects the architecture) 
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